Subspace Embeddings and ℓp-Regression Using Exponential Random Variables

Oblivious low-distortion subspace embeddings are a crucial building block for numerical linear algebra problems. We show for any real p, 1 ≤ p <∞, given a matrix M ∈ Rn×d with n d, with constant probability we can choose a matrix Π with max(1, n1−2/p)poly(d) rows and n columns so that simultaneously for all x ∈ R, ‖Mx‖p ≤ ‖ΠMx‖∞ ≤ poly(d)‖Mx‖p. Importantly, ΠM can be computed in the optimal O(nnz(M)) time, where nnz(M) is the number of non-zero entries of M . This generalizes all previous oblivious subspace embeddings which required p ∈ [1, 2] due to their use of p-stable random variables. Using our matrices Π, we also improve the best known distortion of oblivious subspace embeddings of `1 into `1 with Õ(d) target dimension in O(nnz(M)) time from Õ(d) to Õ(d), which can further be improved to Õ(d) log n if d = Ω(log n), answering a question of Meng and Mahoney (STOC, 2013). We apply our results to `p-regression, obtaining a (1+ )-approximation inO(nnz(M) log n)+poly(d/ ) time, improving the best known poly(d/ ) factors for every p ∈ [1,∞) \ {2}. If one is just interested in a poly(d) rather than a (1 + )-approximation to `p-regression, a corollary of our results is that for all p ∈ [1,∞) we can solve the `p-regression problem without using general convex programming, that is, since our subspace embeds into `∞ it suffices to solve a linear programming problem. Finally, we give the first protocols for the distributed `p-regression problem for every p ≥ 1 which are nearly optimal in communication and computation.